System and method for flexible configuration of a controlling device

ABSTRACT

A system and method whereby an user interface editing device is used to create a user interface for a controlling device, the created user interface including user interface elements that are associated with functional operations of a virtual equivalent of an appliance. After the created user interface is transferred to the controlling device, the virtual equivalent of an appliance is caused to be linked to an intended target appliance whereupon activation of those user interface elements of the graphical user interface that were associated with functional operations of the virtual equivalent of an appliance will cause the controlling device to transmit commands to control corresponding functional operations of the intended target appliance.

RELATED APPLICATION INFORMATION

This application claims the benefit of and is a continuation of U.S.application Ser. No. 17/410,054, filed on Dec. 9, 2021, whichapplication claims the benefit of and is a continuation of U.S.application Ser. No. 16/596,622, Oct. 8, 2019, which application claimsthe benefit of and is a continuation of U.S. application Ser. No.15/632,775, filed on Jun. 26, 2017, which application claims the benefitof and is a continuation of U.S. application Ser. No. 13/673,271, filedon Nov. 9, 2012, which application claims the benefit of and is adivisional of U.S. application Ser. No. 12/629,423, filed on Dec. 2,2009, which application claims the benefit of U.S. Provisional PatentApplication No. 61/201,021, filed on Dec. 5, 2008, and which also claimsthe benefit of and is a continuation-in-part of U.S. application Ser.No. 11/357,681, filed Feb. 16, 2006, which in turn is acontinuation-in-part of U.S. application Ser. No. 11/218,900 (now U.S.Pat. No. 7,266,777), filed on Sep. 2, 2005, which in turn claims thebenefit of U.S. Provisional Patent Application Nos. 60/608,183, filed onSep. 8, 2004, and U.S. Provisional Patent Application No. 60/705,926,filed on Aug. 5, 2005.

Each of these applications is incorporated herein by reference in theirentirety.

BACKGROUND

The following relates generally to controlling devices and, moreparticularly, to a configurable controlling device having an associatededitor program for use in configuring, among other things, the userinterface and functionality of the controlling device.

Editor programs for configuring a controlling device, such as a handheld remote control, are known in the art. For example, U.S. Pat. No.6,211,870 illustrates and describes a controlling device which isprogrammable from a PC using an advanced, object-oriented userinterface. More particularly, multiple user selectable screen objectsmay be created on the PC and transferred to the controlling device. Thescreen objects include screen layout and descriptions of soft keys to bedisplayed on a graphic display of the controlling device, as well ascommands associated with the screen object, the soft keys and/orprogrammable keys on the remote control unit. The user may then selectany of the screen objects once they have been transferred to thecontrolling device to control the operation of various appliances.

Similarly, PCT published application no. WO 00/39772 discloses auniversal, programmable remote control device which has programming thatenables an end-user to customize the remote control device throughediting or programming of the control functionalities of the remotecontrol device. The programming is achieved via a PC. In this manner,the control configuration created via an editor on the PC can bedownloaded into the device. It is additionally disclosed that the PC hasemulator software to test the configuration before downloading. It is tobe understood that WO 00/39772 generally discloses the editor for thePhilips' “Pronto” brand remote control.

Increasingly, such programmable controlling devices are utilized inenvironments which include networked and/or interactive appliances.While these known controlling devices and associated editor programs docontinue to work for their intended purpose in such environments,programming of these controlling devices has become increasingly tediousand in many instances cannot be accomplished without a detailedknowledge of the exact configuration of the target appliance network or,in some cases, without the physical presence of a fully operationalappliance network during the programming process.

SUMMARY

The following generally discloses an editor program for use inconfiguring a user interface of a controlling device, together with aconfigurable controlling device such as a hand-held remote control unit.The editor program allows a user to create on a personal computer fordownloading to the controlling device a graphical user interfacecomprised of user interface pages having icons which icons, whenactivated on the controlling device, cause the controlling device toperform a function, such as to transmit a command to an appliance, tochange the user interface display, etc. The editor program provides,among other things, for selection and placement of icons onto userinterface display pages, assignment of commands to function keys (bothiconic and hard keys), assignment of backgrounds to user interface pagesor groups of pages, creation of icon-activated user-defined commandsequences, pre-rendering of user interface pages prior to downloading ofthe user interface to the controlling device, etc. Advantageously, in anexemplary embodiment the user interface editor and the operatingsoftware of the controlling device may also support the creation andediting of n-state widgets, which are unitary GUI objects comprising oneor more touch activated areas together with one or more display graphics(not necessarily in a one-for-one relationship) and which are capable ofboth initiating parameterized command functions in response to touchinputs, as well as receiving and acting upon status responses fromtarget appliances. Each of the attributes of an n-state widget (toucharea(s), command functions, display data, response behavior) areabstracted from the others and may be independently edited, and not allare required to be present in a given instance of such a widget.

In a further advantageous embodiment, the user interface editor mayprovide for the creation of virtual devices, that is, appliances whichare defined in terms of abstract functionality without any attachment toa physical network address, command transmission format, etc. Using suchvirtual devices, a complete graphical user interface system may bedesigned and implemented independently of actual target appliance andnetwork specifications. Upon download of such GUI programming into acompatible controlling device, the operating software of the controllingdevice may guide a user through a setup process in which virtual devicesare linked to physical appliances present in that controlling device'senvironment. In this manner, graphical user interface system developmentis enabled without necessitating the physical presence of targetappliances, as well as the creation of template GUIs which may be reusedmultiple times in different appliance environments.

The various advantages, features, properties and relationships of thisimproved user interface editor and controlling device system will beobtained from the following detailed description and accompanyingdrawings which set forth illustrative embodiments which are indicativeof the various ways in which the principles thereof may be employed.

BRIEF DESCRIPTION OF THE DRAWINGS

For use in better understanding the user interface editor andconfiguration distribution system and related methods describedhereinafter reference may be had to the following drawings in which:

FIG. 1 illustrates an exemplary home entertainment system in which theexemplary controlling devices may be utilized;

FIG. 2 illustrates a block diagram of components of an exemplarycontrolling device;

FIGS. 3 a and 3 b illustrates exemplary editing systems which may beused to configure and download a controlling device configuration,including a user interface, to an exemplary controlling device;

FIG. 4 illustrates an exemplary home page graphical user interface(“GUI”) for a touch screen of a controlling device;

FIG. 5 illustrates an exemplary device selection page GUI for the touchscreen of a controlling device;

FIG. 6 illustrates an exemplary device control page GUI for the touchscreen of a controlling device;

FIG. 7 illustrates another exemplary device control page GUI for thetouch screen of a controlling device;

FIG. 8 illustrates yet another exemplary device control page GUI for thetouch screen of a controlling device;

FIG. 9 illustrates an exemplary PC-based editor used to create theappearance and functionality of the GUI of FIG. 4 ;

FIG. 10 illustrates an exemplary PC-based editor as used to create theappearance and functionality of the GUI of FIG. 8 ;

FIGS. 11, 12A, 12B, 13A, and 13B illustrate the PC-based configurationprocess of an exemplary multi-state widget;

FIGS. 14A and 14B illustrate the PC-based configuration process of asecond exemplary multi-state widget;

FIG. 15 illustrates the PC-based configuration process of a thirdexemplary multi-state widget;

FIG. 16 illustrates the PC-based configuration of an exemplaryrepetitive polling process for obtaining appliance information to beused by the multi-state widgets illustrated in FIGS. 14 and 15 ;

FIG. 17 illustrates an exemplary virtual device linkage initiation GUIpage for the touch screen of a controlling device;

FIGS. 18A and 18B illustrate an exemplary virtual device linkagedefinition and test GUI pages for the touch screen of a controllingdevice; and

FIGS. 19A, 19B, 19C, and 19D illustrate exemplary touch screen GUI pagesfor the synchronization of the link tables of a pair of controllingdevices.

DETAILED DESCRIPTION

The following discloses a controlling device having a face panel onwhich is carried a user interface activatable to cause transmission ofat least one command to at least one appliance. Turning now to thefigures, wherein like reference numerals refer to like elements, FIG. 1illustrates an exemplary system including controllable appliances, suchas a set top box (“STB”) 104, a DVD player 106, an audioamplifier/receiver 108, a television 102, a lighting fixture 110 andassociated multilevel dimmer 114, and a thermostat 112 as well as acontrolling device 100. The controlling device 100 is capable oftransmitting commands to the appliances, using any convenient IR or RF,point-to-point or networked, protocol to cause the appliances to performoperational functions. While illustrated in the context of a STB 104with DVD player 106, audio system 108, television 102, lighting fixture110 and thermostat 112 it is to be understood that controllableappliances can include, but are not limited to, televisions, VCRs, DVRs,DVD players, cable or satellite converter set-top boxes (STBs),amplifiers, media streaming devices, CD players, game consoles, homelighting, drapery controls, fans, HVAC systems, thermostats, door locks,security systems and cameras, personal computers, etc., and, as such,the instant exemplary disclosures are not intended to be limiting as totype or quantity of controllable appliances or equipment.

Turning now to FIG. 2 , for use in transmitting command codes to one ormore of the appliances, the controlling device 100 of the exemplarysystem may include, as needed for a particular application, a processor200 coupled to a memory device (such as ROM memory 204, RAM memory 202,and/or a non-volatile memory 206), a key matrix 216 (e.g., physicalbuttons, a touch sensitive display with soft keys, or a combinationthereof), an internal clock and timer 222, an IR (and/or RF) transmitter208 for directly issuing commands to controlled appliances, one or moreRF (and/or IR) wireless transmission and reception circuit(s) 210, 212for issuing commands to controlled appliances via a network and/ortransferring data or commands between the controlling device and othernetworked controlling devices or external computing devices such as aPC, a physical input/output interface 224 for use in directlytransferring data between the controlling device and an externalcomputing devices such as a PC, STB, etc., a means 218 to provide visualfeedback to the user (e.g., LCD display or the like, which may underlayall or part of a touch sensitive portion of key matrix 216), a means 220to provide audio feedback (speaker, buzzer, etc.) and a power supply 214all as generally illustrated in FIG. 2 . As will be understood by thoseof skill in the art, the memory device(s) may include executableinstructions that are intended to be executed by the processor 200 tocontrol the operation of the controlling device 100.

The non-volatile read/write memory 206, for example an EEPROM,battery-backed up RAM, Smart Card, memory stick, or the like, may beprovided to store setup data and parameters as necessary. It is to beadditionally understood that the memory devices may take the form of anytype of readable media, such as, for example, ROM, RAM, SRAM, FLASH,EEPROM, Smart Card, memory stick, a chip, a hard disk, a magnetic disk,and/or an optical disk. Still further, it will be appreciated that someor all of the illustrated memory devices 202, 204, and 206 may bephysically incorporated within the same IC chip as the microprocessor200 (a so called “microcontroller”) and, as such, they are shownseparately in FIG. 2 only for the sake of clarity.

To cause the controlling device 100 to perform an action, thecontrolling device 100 is adapted to be responsive to events, such as asensed user interaction with the key matrix 216, receipt of a data orsignal transmission, etc. In response to an event appropriateinstructions within the memory may be executed. For example, when acommand key is activated on the controlling device 100, the controllingdevice 100 may retrieve a command code corresponding to the activatedcommand key from memory 204 or 206 and transmit the command code to adevice in a format recognizable by the device. It will be appreciatedthat the instructions within the memory can be used not only to causethe transmission of command codes and/or data to the appliances but alsoto perform local operations. While not limiting, other local operationsthat may be performed by the controlling device 100 include execution ofpre-programmed macro command sequences, displaying information/data,manipulating the appearance of a graphical user interface presented on alocal LCD display 218, etc., and in an exemplary embodiment illustrativeof the instant inventive concepts, managing the linkage between virtualdevices defined by a GUI editing program and real devices present in thecontrolled environment. For convenience and economy of developmenteffort, the software programming of controlling device 100 may utilizean underlying operating system such as, for example, Microsoft's“Windows CE” or “Windows Mobile” brand operating systems.

As contemplated in the above referenced and related U.S. patentapplication Ser. Nos. 11/357,681 and 11/218,900 and provisionalapplications 60/608,183 and 60/705,926, the graphical user interface(“GUI”) and certain functionalities of controlling device 100 may bedefined via a software based editing tool 300 which may be supplied asan application program to be installed on a PC 302 running an operatingsystem, for example, a Microsoft “Windows” brand operating system, asgenerally illustrated in FIGS. 3 a and 3 b and described in furtherdetail hereinafter. In the illustrative examples that follow, it will beappreciated by those skilled in the art that development tools such asMicrosoft's Visual Studio, the C# programming language, and variousthird party libraries may be used to facilitate creation of the softwarecomprising exemplary editing tool 300 and exemplary controlling device100 GUI and functionality.

Editor application 300 may be offered by the manufacturer of thecontrolling device 100 on a CD ROM, for download from a Web site, etc.,as appropriate for installation on a PC of the user's choice. Once theeditor application is installed on the user's PC 302, the controllingdevice GUI may be created or revised using the editor application,stored locally as a file 310 on PC 302 and/or caused to be downloadedinto controlling device 100 via a hardwired connection 304, a wirelesslink 306 (e.g., WiFi, Bluetooth, Zigbee, etc.) or any other convenientmeans. Additionally, it will be appreciated that the editor application300, although primarily resident on the user's local PC 302, may also beadapted to access additional data items from remotely located serversvia the Internet 308, from appliances linked to the PC 302 via a homenetwork, etc. Examples of such items may include, without limitation, IRcommand codes (e.g., to allow for support of new appliances), data whichindicates operations supported by an appliance, device model numbercross-references (e.g., for entering into the controlling device forset-up purposes as disclosed in, for example, U.S. Pat. No. 6,587,067),operational software updates for controlling device 100, etc. It willalso be appreciated that in such an environment data may also beuploaded from PC 302 to a centralized repository, e.g., a remotelylocated, Internet accessible server. Such uploaded information mayinclude, for example, current user configurations, learned IR code data,etc., and may be comprised of or derived from data stored locally on PC302 (for example, file 310) and/or data retrieved from controllingdevice 100 during the times controlling device 100 is coupled to PC 302.

Certain aspects of the operation of exemplary controlling device 100will now be discussed in conjunction with FIGS. 4 through 8 . In thiscontext, as will be appreciated by those familiar with the relevant artand/or with the previously referenced parent and U.S. provisionalapplications, the actual appearance and functionality of all the GUIpages in controlling device 100 represent only one instance of theoutput of editor application 300. It will thus be understood that theGUIs and associated functionality presented herein are by way of exampleonly and not intended to be limiting in any way.

Controlling device 100 may include both a touch activated LCD screen 218with soft keys (or other form of touch panel) and several groups of hardbuttons 414, 416, 418, 420. The hard buttons groups might comprise, forexample, a volume control group 416 (e.g., volume up, down, and mute), achannel changing group 418 (e.g., channel up, down, and return), anavigation group embodied in disk 414 (e.g., for menu navigation andselection including up, down, left, right, and enter/select), and/or arow of programmable keys 420 (e.g., keys for supporting macros or otherto-be-configured functions).

Upon start of operation, or any time the “Home” button 422 (e.g., on theside of the device) is activated, an exemplary Home Page GUI 400 may bepresented within the display. The illustrated, exemplary home Page 400includes five touch-activated buttons. By way of example only, touchingicon 402 may be used to initiate the activity of watching cable TV bycausing controlling device 100 to transmit the commands required topower on cable STB 104, power on TV 102, select the TV input to whichthe cable STB is connected, and then cause the controlling device GUI totransition to a page (e.g., display having soft keys, an EPG, or thelike) from which cable STB channel selection may be input (for furthertuning, retrieving related content information, etc.). Touching icon 404may be used to similarly cause the controlling device to place theentertainment system into a condition suitable for watching a DVD movie,while touching icon 406 may be used to cause the controlling device toplace the system into a condition suitable for listening to music.Touching icon 410 may be used to cause the controlling device totransition to another page 800 of the GUI (an example of which is shownin FIG. 8 ) from which home automation functions such as control ofillumination levels, room temperature, etc. may be initiated. Touchingicon 408 may be used to cause the controlling device to transition toanother page 500 of the GUI (an example of which is shown in FIG. 5 )from which individual appliance control pages may be called up, i.e.,navigated to, using icons 502 (cable STB), 504 (TV), 506 (DVR player),508 (audio receiver), 510 (lighting), and 512 (thermostat). Examples ofindividual appliance control pages are shown in FIG. 6 (GUI page 600having soft keys for use in controlling receiver input selectioncommands), FIG. 7 (GUI page 700 having soft keys for use in controllingDVD transport functions), and FIG. 8 (GUI page 800 comprising a pair ofn-state widgets 802 and 804 for controlling multi-level dimmable lamp110 and thermostat 112 respectively.) In this way it will be understoodand appreciated that a plurality of commands and/or actions may beperformed both locally on controlling device 100 and remotely on thevarious appliances under the control of controlling device 100 inresponse to a single or multiple interaction(s) by a user with the keymatrix 216 of controlling device 100.

Turning now to FIG. 9 , the operation of the editing program 300 used tocreate the above-disclosed, exemplary GUI pages and associatedfunctionality will be described in further detail. For this purpose, thePC screen display of the editor application may be divided into severalwindows or panels, each having a specific purpose. By way of example,the panels may be as follows:

Main project panel 902, used to display the current GUI page beingedited (Home Page 400 in this illustration) together with arepresentation 922 of the hard keys (keys 414 through 420) available ontarget controlling device 100;

Project View panel 904, used to display all currently defined GUI pagesin a tree structure form (which may, as depicted in the exemplaryembodiment shown, have collapse [−] and expand [+] functionalityincluding selective expansion of individual nodes and/or a collapseall/expand all feature) where the GUI page to be edited may be selected(e.g., by clicking on a link) from within the tree structure list andwherein the GUI page being displayed in the Main project panel 902 maybe indicated by a highlight 918 (the Home Page in the instantillustration);

Properties panel 906, used to display a list of (and allow editingof—for example by text entry, selection from drop down menus, etc.) theproperties (such as the caption text and font attributes, symbolposition, button type, etc.) associated with a presently selected GUIicon or hard key image within Main project panel 902 (GUI icon or softkey 404′ with label “DVD Movie” in this illustration, as indicated bythe highlight (dotted line) around icon 404′ displayed in Main projectpanel 902);

Gallery panel 908, used to display graphic images which may be draggedand dropped onto the GUI pages being edited wherein the Gallery choicesmay include sets of icons for use as buttons, page backgrounds, symbolsfor labeling buttons, key groups (to allow a group of related key icons,e.g. a numeric pad, to be dragged into place in a single operation), toallow pre-defined themes to be applied to single pages or groups ofpages, or in an exemplary embodiment to allow home control interfaceitems such as n-state widgets to be dragged and dropped onto GUI pagesbeing edited, and wherein the Gallery in use (when multiple, organizedGalleries are provided) is selected via tabs 914 according to theexemplary embodiment shown;

Actions panel 910, used to display a list of (and allow editing of—forexample by dragging and dropping to change the order, by deletingselected items, etc.) the actions to be performed by controlling device100 when the currently selected icon (in the Main panel 902) isactivated by a user when the user interface is provided to thecontrolling device (e.g., in the example presented, it can be seen thatactivating the “DVD Movie” icon 404′ will: (1) transmit a “Power On”command to DVD player 106, (2) transmit a “Power On” command to TV set102, (3) wait one second for the devices to stabilize, then (4) transmita “Component 2” input selection command to TV 102, (5) transmit “DVD”input selection command to Audio Receiver 108, and finally (6) jump toGUI page 700 corresponding to the DVD transport controls (illustrated inFIGS. 7 and 10 ); and

Devices panel 912 which is preferably used to display a listing of allremote control commands available for each of the appliances setup to becontrolled by controlling device 100 wherein the list of commandablefunctions for a given appliance to be displayed may be selected from adrop down list 916 (which in the illustrative example would comprise aTV 102, a cable STB 104, a DVD player 106, and an audio receiver 108)and wherein the commandable functions so displayed may be assigned toany icon displayed in main project panel 902 by simply clicking anddragging a commandable function icon to, for example, a desired locationwith a listing of functions displayed in the Actions panel 910 and/orover an icon displayed in the Main panel (where it would be added, forexample, to the top or bottom of the listing of commands within theAction panel display).

The operation of an exemplary editing program 300 used to create theabove-disclosed, illustrative “Home control” GUI page 800 will be nowdescribed in further detail with reference to FIGS. 10 through 13 .After selection of the appropriate GUI page from the page tree ofProject View panel 904 (or creation of a new page via “add page” tab1014 if no such page yet exists), selection of Home Control tab 1002 ofthe gallery panel may result in the display of various home controlwidgets 908′ available for use in configuring the controlling deviceGUI. In an illustrative example, a 4-state lighting control widget 1004and thermostat control widget 1006 may be selected and dragged onto theGUI page currently under definition, at locations 1008 and 1010respectively. Thereafter, these widgets may be associated withappliances to be controlled by selecting an appropriate appliance indevices panel 912 to be dragged and dropped onto each widget. By way ofillustration, in the example presented a template multilevel lightingdevice 1012 has been selected and associated with widget icon 1008. Inthis regard, in certain instances such devices may take the from ofgeneric templates with the final linkage of GUI widgets to specificphysical appliances deferred until such time as the completed GUIconfiguration is downloaded and installed on a specific controllingdevice, as will be described in greater detail in a subsequent sectionof this document.

As mentioned earlier, an n-state widget, such as 4-state lightingcontrol widget 1008, may comprise multiple touch areas, actions, images,and status responses all of which are independently variable. Onceplaced on a GUI page as described above, in an exemplary embodiment ofediting program 300 double clicking image 1008 may initiate a widgetconfiguration wizard 1100 superimposed on the standard PC screen display1150 of editor application 300 as illustrated in FIG. 11 . Such a wizardmay include without limitation tabs 1102 and 1104 for configuration ofcontrol (widget outputs) and status feedback (widget input)functionalities respectively. When control tab 1102 is selected asillustrated in FIGS. 11 and 12 , wizard display 1100 may include alisting 1106 of possible states, four in this specific example, fromwhich the state currently under configuration may be selected, state onein this illustration. Configuration input areas may comprise a field1008 into which may be entered a user-friendly descriptive name for thestate; a drop-down menu 1110 for selection of a parameter value to betransmitted to the target appliance upon selection of this state, zeroor “Off/Disable” in this instance, it being noted that this value may bea generic abstraction which may be converted to a real appliance commandvalue at a later point when the widget is linked to a target appliance;a set of entries 1112 for defining the portion of the overall widgettouch area which is to be associated with this state, expressed in anexemplary embodiment in terms of width and height, in pixels, and an X,Ystarting coordinate relative to the top left corner of the overallwidget area; and an image 1114 to be displayed upon activation of thistouch area, which may be selected from a user library of images via abrowse function 1116 as desired.

Turning now to FIG. 12 , additional steps in configuring widget 1008 areillustrated (for clarity, background display 1150 has been omitted fromall further illustrations of the widget configuration wizard). Asillustrated at 1200, the third state 1202, labeled “Mid” is configuredto transmit a parameter value of “67” upon activation of the relevanttouch area, while as illustrated at 1220 the fourth state 1222 labeled“Full” is configured to transmit a parameter value of “255-On/Enable”upon activation of the relevant touch area.

Turning now to FIG. 13 , upon activation of the “Status Feedback” tab1104 of exemplary widget configuration wizard 1100, an exemplary inputscreen 1300 may be displayed to permit configuration of the response ofn-state widget 1008 to status feedback messages received from targetappliances. As illustrated at 1300, the state zero response, selected at1302, is invoked by an returned state value in the range bounded by theminimum/maximum values selected from drop down menus 1304 and 1306. Inthis illustrative example, it can be seen that the minimum/maximumvalues are identical, i.e., this response, being the icon display ofimage 1308, may only be invoked upon receipt of a returned status valueof exactly zero, or off/disabled, from the target multilevel switch. Asillustrated at 1320 however, it can be seen that the state threeresponse selected at 1322 may be invoked by any returned status value inthe range “50” through “90” to cause display 1328.

In some embodiments, certain n-state widgets may be defined which do notinclude all the attributes described in the previous example, i.e.,image, touch control output, status input, and status response. By wayof illustration, an exemplary current temperature display widget of athermostat control, such as 1018 in FIG. 10 , may include only statusinput and image response functionality. Definition of the behavior ofsuch a widget may be as illustrated in FIG. 14 , noting that the setupwizard display in this instance lacks a “control” setup tab such as seenat 1102, and presents only the “status” setup tab 1104′ since no controlfunctionality is associated with this exemplary widget. With referenceto the setup wizard displays of FIG. 14 , the behavior of the exemplarytemperature display widget has been defined at field 1404 to display theactual numeric value returned by the thermostat appliance when thisvalue is in the range of “32” to “99” as defined at 1402, and to displaythe text “Error” 1410 should the returned value be either higher 1408 orlower 1406 than these values, as illustrated 1412 in the case of therange of lower values. As will be appreciated, in order to maintain acurrent representation of the status value displayed, the implementationof such a widget in controlling device 100 may be required to poll thetarget appliance for an updated status from time to time. Theconfiguration and operation of such a polling feature will be discussedin further detail hereafter.

Turning now to FIG. 15 , by way of further illustration an exemplarysecond instance of a temperature display n-state widget 1502 may beplaced on another page 1500 of the GUI page suite, associated with thesame thermostat appliance, and defined 1504 in the same manner asdescribed above, for example to serve as an informational display topermit a user of controlling device 100 to check current roomtemperature directly from an activities selection page 1500 as shown. Inthis example however, rather that a numerical value, the widget isdefined to simply display “OK” if the reported temperature is in therange of “72” to “78” degrees.

As is apparent from the above, multiple n-state widgets may be definedto monitor and react to the same status of a target appliance.Accordingly, in some embodiments the required appliance pollingfunctionality of controlling device 100 may be implemented as a singleinstance associated with the target appliance definition, andindependent of the widgets which utilize the response data. In thismanner a single polling sequence may retrieve the required status dataon a periodic basis and cache the response value(s) for use as needed byparticular widgets, e.g., when displayed, thereby minimizing overhead onthe network or other communication link(s) used for appliance commandand status messaging. In an exemplary embodiment, the polling parametersmay thus be configured in GUI editor application 300 as a characteristicof the target appliance representation, and not as a characteristic ofindividual widgets. By way of example, clicking on an “edit device” icon1016 in devices panel 912 may invoke an exemplary device propertiesconfiguration wizard 1600 for the current device, as illustrated in FIG.16 . Device configuration wizard 1600 may include a drop down menu 1602from which an appropriate polling interval may be selected. As before,these parameters may be associated with a generic virtual device whichis to be linked to an actual appliance only upon deployment of the GUIprogramming to a controlling device, for example controlling device 100.

When a user of exemplary editor program 300 has completed creationand/or modification of a GUI configuration, the GUI configuration may besubsequently transferred to a controlling device 100 either via aphysical connection (e.g., a cable 304, a docking station, etc.) or viaa wireless connection 306 (e.g., WiFi, Bluetooth, Z-wave, Zigbee, or anyother suitable protocol), for example by clicking “Download project” tab924. Prior to downloading, editor program 300 may pre-process certainelements of the project file 310 data into a data structure 311 whichmay be in a format more suitable for direct use within controllingdevice 100, as described in greater detail in previously referenced U.S.patent application Ser. No. 11/218,900.

As mentioned earlier, in certain self-configuring networks such as forexample a Z-wave wireless mesh network for home control and automationbased on components available from Zensys Inc., final appliance networkaddresses, etc., may only be determinable by a controlling device at thetime that the controlling device is physically introduced into thenetwork. Accordingly, in some embodiments a GUI configuration downloadedinto an exemplary controlling device 100 may include generic virtualdevices created as placeholders during the editing process. Turning nowto FIG. 17 , upon introduction of such an exemplary controlling device100 into the network ecosystem in which it is to operate aninitialization screen 1700 may be accessed to allow establishment oflinkage(s) between virtual devices configured in the GUI programming ofcontrolling device 100 and the physical network nodes which correspondto target controlled appliances in the ecosystem. In an exemplaryembodiment, a user of controlling device 100 may be presented with touchicons 1704 activatable to initiate a link management wizard, 1702activatable to initiate transfer of preexisting link assignments fromanother controlling device already present in the ecosystem, and 1706 todefer establishment of linkages to later time. As will be appreciated,if this latter alternative 1706 is selected any currently unlinkedvirtual devices will be non-functional until such future time as linkageis completed. Such non-operational status may be represented in the GUIpages of controlling device 100 by grayed-out or otherwise alteredicons, error messages on entry to those pages or upon icon actuation,visible or audible warning upon icon actuation, etc., or any combinationthereof as appropriate for a particular embodiment.

Upon activation of “Link” icon 1704, in an exemplary embodiment a usermay be presented with a display 1800 as illustrated in FIG. 18 . (Forbrevity, only the screen contents and not the full outline ofcontrolling device 100 are presented in this and subsequentillustrations.) Such a display may comprise an area 1802 in which may bepresented a list of virtual devices present in the GUI configuration ofcontrolling device 100 together with their current linked/unlinkedstatus, together with an area 1804 in which may be presented a list ofunlinked physical nodes present in the current ecosystem. The list ofvirtual devices presented in area 1802 may include indications 1806,1808 representative of the current linked (1806) or unlinked (1808)status of each virtual device, together with an indication 1820 of theassociated physical appliance node for those virtual devices which arelinked. In the example illustrated, it can thus be seen that themultilevel lighting virtual device, corresponding for example to GUIicon 1008 of FIG. 10 , has been linked to physical node “4” of thecurrent Z-wave ecosystem. Checkboxes 1810, 1811 and 1812 may be providedfor user selection of individual items listed in the two areas. In orderto link a virtual device to physical node, a user may touch one checkboxin each of the respective lists, for example 1811 and 1812, followed byactuation of the “Link” icon 1814.

Alternatively, to cancel an existing linkage a user may touch thecheckbox associated with an already configured virtual device, forexample checkbox 1810 corresponding to the link 1806, followed byactuation of the “Break” icon 1816. In confirmation of such an action,the linkage indicator 1806 may be changed to an “unlinked” symbolsimilar to that illustrated at 1808 and an entry corresponding to thenewly-detached physical node “4” added to the list of availableecosystem nodes displayed in area 1804. In some embodiments, a “Reset”icon 1822 may also be provided, actuation of which may break allexisting links—to be used, for example, when relocating a controllingdevice to an entirely new environment.

Before a link between a virtual equivalent of an appliance and anintended target appliance is finalized for use within an operationalconfiguration of the controlling device 100, a user may be afforded anopportunity to ascertain if controlling device 100 is communicating withan intended target appliance or an appropriate appliance node eitherbefore or after a virtual device to physical node linkage isestablished. In an exemplary embodiment this may be achieved byselection of either an already-linked virtual device from the list inarea 1802 (e.g., Multilevel Lighting node 1810) or selection of oneunlinked item from each of areas 1802 and 1804 (e.g., Generic Thermostat1811 and Z-wave ecosystem node “3” 1812), followed by actuation of“Test” icon 1818. In the embodiment illustrated, this may result indisplay of a page 1850 which lists the features and status of theselected appliance (node “4”, Multilevel Lighting, in the illustratedcase.) From this display, a user may ascertain the node ID 1864 andassociated (or to be associated) virtual device 1862, the type(s) ofservice 1854 available from this appliance node, and the current statusof the appliance node 1870. Further, the user may exercise the applianceby selecting a parameter value from drop-down table 1860 and actuatingthe “Set” icon 1858 to cause transmission of that value to the appliancenode. Subsequently, the changed status of the appliance may be verifiedby actuation of the “Get” icon 1856 which may cause the status display1870 to be updated by retrieval of a new value from the appliance node.By way of specific example, if multilevel lighting node “4” correspondedto dimmer outlet 114 of FIG. 1 , then upon entry into the test theillustrated displayed status 1870 would indicate that lamp 110 wascurrently set to 60% of maximum brightness. If the illustrated parameter1860 (“0”) were then to be transmitted to dimmer outlet 114 by actuationof the “Set” icon 1858, lamp 110 should be observed to turn off and uponsubsequent actuation of the “Get” icon 1856, reported status 1870 shouldbe observed to change to “0/Off”.

In the event that appliance services or responses are not as expected ordesired, if an already-linked virtual device/appliance node pair wasunder test, the link may be immediately broken by actuation of “Break”icon 1868. After completion of all desired testing, controlling device100 may be returned to its previous state (i.e., screen 1800) byactuation of “Back” icon 1868.

In the event that a similar or compatible controlling device has alreadybeen configured with a set of virtual device to physical device linkagesappropriate for the current ecosystem, upon introducing an additionalcontrolling device 100 into the environment a user may select the“synchronize” option 1702, which mode may be used to transfer a copy ofthe exiting linkage assignments from the previously configured othercontrolling device to the newly-introduced controlling device. In orderto accomplish this, both controlling devices may placed into the linkageinitialization mode illustrated in FIG. 17 , for example from a“settings” menu on the home pages of each, and the “Sync” icon 1702actuated on both. With reference to FIG. 19 , both controlling devicesmay then display an introductory screen 1900 comprising icons 1902, 1904activatable to place one of the controlling devices into a mode forsending linkage data and the other controlling device into a mode forreceiving linkage data. Upon actuation of the respective icons on therespective controlling devices, the controlling device placed into asend mode may display screen 1910 while the controlling device placedinto receive mode may display screen 1920. With reference to display1910, an exemplary sending controlling device may present a user prompt1912 requesting initiation of the synchronization process by activatinga “Sync” icon 1914. Upon such initiation of the synchronization process,the sending controlling device may display a message 1916 requestingthat the user activate the receive mode on another controlling device,and may thereafter begin polling a wireless local network (e.g., WiFi,Z-wave, Bluetooth, etc.) for a receive ready response from a likecontrolling device. With reference to display 1920, an exemplaryreceiving controlling device may present a user prompt 1922 requestinginitiation of reception by activating a “Receive” icon 1924. Uponentering into the receive mode, the exemplary controlling device maybegin to monitor a wireless local network for an enquiry from a likecontrolling device seeking to transfer linkage data. Upon detection ofsuch a signal, an affirmative response may be issued by the receivingcontrolling device, after which data transfer may begin. During thetransfer process, the receiving controlling device may display a statusmessage 1926 indicating to the user that the transfer is in progress.Upon successful completion of the transfer, both controlling devices maydisplay a completion message as illustrated by screen 1930, togetherwith an icon 1932 activatable to end the synchronization process andreturn each controlling device to its previous operational mode.

While various concepts have been described in detail, it will beappreciated by those skilled in the art that various modifications andalternatives to those concepts could be developed in light of theoverall teachings of the disclosure. For example, while the embodimentspresented above are described in the context of universal remotecontrols (i.e., controlling devices capable of commanding the operationof multiple classes of appliances devices from multiple manufacturers)as being most broadly representative of controlling devices in general,it will be appreciated that the teachings of this disclosure may beequally well applied to other controlling devices of narrowercapability, and also to any general or specific purpose device requiringa visual interface (i.e., display screens, signage devices,teleprompters, etc.) without departing from the spirit and scope of thepresent invention. Still further, it will be appreciated that the userinterfaces described herein need not be limited to controlling devicesbut can be utilized in connection with any device having input elementswherein it is desired to convey information concerning such inputelements. For example, the user interface may be utilized with devicessuch as calculators, phones, appliances, etc. having input elementshaving associated information conveying images in the form ofalphanumeric and/or symbolic labels. It will also be understood thatmodification, editing, or updating of configuration settings may beperformed either by a user or though any automated computing processesas are well known in the art. As such, the particular concepts disclosedare meant to be illustrative only and not limiting as to the scope ofthe invention which is to be given the full breadth of the appendedclaims and any equivalents thereof.

All documents cited within this application for patent are herebyincorporated by reference in their entirety.

What is claimed is:
 1. A method for creating a configuration for acontrol device, comprising: automatically determining, with at least oneprocessor, a plurality of devices controllable with the control device,wherein each device of the plurality of devices is associated with atleast one device category of the following device categories: a mediasource device category, a receiver device category, a display devicecategory, an audio device category, or any combination thereof;displaying, on a first graphical user interface, a first set of deviceoptions corresponding to a first set of devices of the plurality ofdevices, the first set of devices associated with the media sourcedevice category; displaying, on the first graphical user interface, asecond set of device options corresponding to a second set of devices ofthe plurality of devices, the second set of devices associated with atleast one of the display device category and the audio device category;receiving, through the first graphical user interface, a selection of afirst device from the first set of device options and a second devicefrom the second set of device options; associating the first device andthe second device with an activity object; and in response to anactivation of the activity object via the control device, using theassociation of the first device and second device with the activityobject to automatically determine, with the control device, a set ofcontrol signals configured to control the first device and the seconddevice.
 2. The method of claim 1, further comprising: displaying, withthe at least one processor, a plurality of input options associated withat least one of the first device and the second device; receiving aselection of an input option from the plurality of input options;associating, with at least one processor, the input option with theactivity object; and further using the association of the first inputoption with the activity object to automatically determine, with thecontrol device, the set of control signals configured to control thefirst device and the second device.
 3. The method of claim 1, whereinthe at least one processor comprises a processor of a computing deviceseparate from the control device.
 4. The method of claim 3, wherein theprocessor of the computing device is configured to display the firstgraphical user interface and receive the selection of the first deviceand the second device.
 5. The method of claim 1, further comprising:automatically transmitting, with the control device, a power activationsignal to each of a subset of devices of the plurality of devices, thesubset of devices associated with the activity object and including thefirst device and the second device.
 6. The computer-implemented methodof claim 5, further comprising: displaying, on the control device, aplurality of input options associated with at least one device of thesubset of devices; receiving a selection of an input option from theplurality of input options; and transmitting at least one control signalof the set of control signals based on the input option.
 7. The methodof claim 6, wherein the plurality of input options comprises aselectable toggle option, the method further comprising: in response toactivation of the selectable toggle option, transmitting, with thecontrol device, at least one control signal configured to switch aninput of the at least one device from a first input to a second input,wherein the second input comprises the selected input option.
 8. Themethod of claim 1, further comprising generating, on the control device,a device control interface comprising a plurality of selectable controloptions, wherein at least one first selectable control option of theplurality of selectable control options, when selected, causes thecontrol device to determine a first control signal configured to controlthe first device, and wherein at least one second selectable controloption of the plurality of selectable control options, when selected,causes the control device to determine a second control signalconfigured to control the second device.
 9. The method of claim 1,further comprising: displaying, on the first graphical user interface, athird set of device options corresponding to a third set of devices ofthe plurality of devices, the third set of devices associated with thereceiver device category; receiving, through the first graphical userinterface, a selection of a third device from the third set of deviceoptions; associating the third device with the activity object; andfurther using the association of the third device with the activityobject to automatically determine, with the control device, the set ofcontrol signals configured to control the first device and the seconddevice.
 10. The method of claim 9, further comprising: displaying, onthe first graphical user interface, a fourth set of device optionscorresponding to a fourth set of devices of the plurality of devices,the fourth set of devices associated with at least one of the displaydevice category and the audio device category; receiving, through thefirst graphical user interface, a selection of a fourth device from thefourth set of device options; associating the fourth device with theactivity object; and further using the association of the fourth devicewith the activity object to automatically determine, with the controldevice, the set of control signals configured to control the firstdevice and the second device.
 11. A system for creating a configurationfor a control device, comprising a processor configured to:automatically determine a plurality of devices controllable with thecontrol device, wherein each device of the plurality of devices isassociated with at least one device category of the following devicecategories: a media source device category, a receiver device category,a display device category, an audio device category, or any combinationthereof; display, on a first graphical user interface, a first set ofdevice options corresponding to a first set of devices of the pluralityof devices, the first set of devices associated with the media sourcedevice category; display, on the first graphical user interface, asecond set of device options corresponding to a second set of devices ofthe plurality of devices, the second set of devices associated with atleast one of the display device category and the audio device category;and receive, through the first graphical user interface, a selection ofa first device from the first set of device options and a second devicefrom the second set of device options; associate the first device andthe second device with an activity object; and whereupon, in response toactivation of the activity object via the control device, the controldevice will use the association of the first device and second devicewith the activity object to automatically determine a set of controlsignals.
 12. The system of claim 11, wherein the processor is furtherconfigured to: display a plurality of input options associated with atleast one of the first device and the second device; receive a selectionof an input option from the plurality of input options; and associatethe input option with the scene object, and wherein the control devicewill use further the association of the input option with the activityobject to automatically determine the set of control signals.
 13. Thesystem of claim 11, wherein the processor is further configured to:display, on the first graphical user interface, a third set of deviceoptions corresponding to a third set of devices of the plurality ofdevices, the third set of devices associated with the receiver devicecategory; receive, through the first graphical user interface, aselection of a third device from the third set of device options; andassociate the third device with the scene object, and wherein thecontrol device will use further the association of the third device withthe activity object to automatically determine the set of controlsignals.
 14. The system of claim 13, wherein the processor is furtherconfigured to: display, on the first graphical user interface, a fourthset of device options corresponding to a fourth set of devices of theplurality of devices, the fourth set of devices associated with at leastone of the display device category and the audio device category;receive, through the first graphical user interface, a selection of afourth device from the fourth set of device options; and associate thefourth device with the scene object, wherein the control device will usefurther the association of the fourth device with the activity object toautomatically determine the set of control signals.
 15. A computerprogram product for creating a configuration for a control device,comprising at least one non-transitory computer-readable mediumincluding program instructions that, when executed by a processor, causethe processor to: automatically determine a plurality of devicescontrollable with the control device, wherein each device of theplurality of devices is associated with at least one device category ofthe following device categories: a media source device category, areceiver device category, a display device category, an audio devicecategory, or any combination thereof; display, on a first graphical userinterface, a first set of device options corresponding to a first set ofdevices of the plurality of devices, the first set of devices associatedwith the media source device category; display, on the first graphicaluser interface, a second set of device options corresponding to a secondset of devices of the plurality of devices, the second set of devicesassociated with at least one of the display device category and theaudio device category; receive, through the first graphical userinterface, a selection of a first device from the first set of deviceoptions and a second device from the second set of device options; andassociate the first device and the second device with an activityobject; whereby, in response to activation of the activity object viathe control device, the controlling device is caused to use theassociation of the first device and second device with the activityobject to automatically determine a set of control signals.